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Abstract. The paper is devoted to the generalization of Lusztig's g-analog of weight multiphcitios to 
the Lie superalgebras 0((?i, m) and spo(2n, M). We define such g-analogs Kx^^{q) for the typical modules 
and for the irreducible covariant tensor g[(n, m,)-modules of highest weight A. For gl(n, m), the defined 
polynomials have nonnegative integer coefficients if the weight fi is dominant. For spo(2n, M), we show 
that the positivity property holds when /i is dominant and sufficiently far from a specific wall of the 
fundamental chamber. We also establish that the ^-analog associated to an irreducible covariant tensor 
gl(n, m)-module of highest weight A and a dominant weight fi is the generating series of a simple statistic 
on the set of semistandard hook-tableaux of shape A and weight /i. This statistic can be regarded as a 
super analog of the charge statistic defined by Lascoux and Schiitzenberger. 



1. Introduction 

There has been considerable interest recently in defining and studying g-analogs of various coefficients 
in the representation theory of semisimple Lie algebras and, more generally, of Kac-Moody algebras. An 
important such g-analog is the one defined by Lusztig [20\ for the dimension of the space of weight /x in 
the irreducible representation of a semisimple Lie algebra with highest weight A. This g-analog is usually 
denoted by K\^^{q), and is known as a Kostka-Foulkes polynomial. Its importance is highlighted by its 
occurence in various contexts beside the original definition. For instance, it was shown to be an affine 
Kazhdan-Lusztig polynomial (and therefore has positive integer coefficients), it gives the expansion of 
a Hall-Littlewood polynomial in the basis of irreducible characters, it encodes the Brylinski-Kostant 
filtration of weight spaces, and is closely related to the so-called energy function in the representation 
theory of affine algebras |l8j. Other g-analogs studied recently include the generalization of Lusztig's 
g-analog to symmetrizable Kac-Moody algebras [5S] , and various g-analogs of branching coefficients for 
semisimple Lie algebras (see [T7] and the references therein). 

In this paper, we define and study a generalization of Lusztig's g-analog of weight multiplicities to the 
most fundamental Lie superalgebras, namely the general linear superalgebras 0l(n, m) and the orthosym- 
plectic superalgebras spo(2n, M). A substantial interest in Lie superalgebras comes from mathematical 
physicists, due to various physical interpretations. In his fundamental paper, Kac classified the 
simple finite dimensional Lie superalgebras that are not Lie algebras. He also gave a formula for the 
characters of the finite dimensional irreducible representations of these superalgebras which are known 
as typical. Since Kac's paper, the investigation centered on character formulas for the atypical repre- 
sentations, and on extending Kashiwara's theory of crystals p3] to Lie superalgebras. 

Our g-analog of weight multiplicities for the typical representations of Lie superalgebras is based on 
a natural quantization of Kac's character formula. More precisely, it is based on a g-partition function 
which, in a certain sense, is defined in precisely the same way as its counterpart for semisimple Lie 
algebras, on which the definition of Lusztig's g-analog is based. We also define a g-analog for the 
irreducible covariant tensor modules via a quantization of a character formula due to Berele-Regev and 
Sergeev. The positivity of these g-analogs is proved by reducing it to that of Lusztig's g-analog. 
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It is well-known that Lusztig's g-analog for the general linear groups is expressed combinatorially via 
the Lascoux and Schiitzenberger charge statistic on semistandard Young tableaux |TB]. We derive here 
a similar statistic on the set of semistandard hook-tableaux introduced in p], and show that it can be 
used to express the g-analog associated to an irreducible covariant tensor rn)-module. 

Our g-analog K\^((i) for the orthosymplectic algebras does not have the positivity property in general. 
Nevertheless, we prove that the positivity holds in an important special case, which is related to a certain 
stabilization phenomenon occuring when the dominant weight /i is sufficiently far from a specific wall 
of the fundamental chamber. It is interesting to note that the stabilized version of Lusztig's g-analog 
for the orthogonal and symplectic algebras can be computed via a combinatorial algorithm; the reason 
is a connection with the energy function on afhne crystals, which is explained in [18]. In general, 
there is no known combinatorial formula for Lusztig's g-analogs in types B — D; in particular, there 
is no generalization of the Lascoux-Schiitzenberger charge. This suggests that our g-analogs for the 
orthosymplectic algebras will also be hard to compute combinatorially. 

It would be interesting to see whether any of the structures related to Lusztig's g-analog (affine Hecke 
algebras, Hall-Littlewood polynomials, the Brylinski-Kostant filtration, the energy function) could be 
defined for Lie superalgebras. If so, we expect them to be related in the same way to our g-analog. 

Acknovirledgements. We are grateful to Georgia Benkart, Jonathan Brundan, and Ronald King for 
helpful discussions, as well as to the anonymous referee for the careful reading of the manuscript and 
the suggestions. 

2. Background 

We recall in this section, some background on classical Lie superalgebras. The reader is referred to 
[S] and [TT] for a more detailed exposition. 

2.1. The root systems for the Lie superalgebras Ql{n,m) and spo{2n, M). Let n and m be two 

positive integers. Let g = 0offi0i be one of the Lie superalgebras m), spo(2n, 2m-|-l), or spo(2n, 2?7i) 
over C. Let t) be a Cartan subalgebra of go. The root system of g is graded so that the set of positive 
roots A+ = A(j" U A'^ is the disjoint union of the set of positive even and positive odd roots. Let 
{Sn, . . . , Sj, Si, . . . , S,n} be a basis of f)*. We denote by ( ■ , • ) the inner product defined on f)* by 

((5-, := (5ij , {Sr, Ss) := -Sr^s , and ( (5-, (5^ ) := , 

where S^^v is the usual Kronecker symbol. 

Among the different choices of a set of simple roots, one is called distinguished; the corresponding even 
roots are those of go, and the odd one is the lowest weight corresponding to the action of gi on go. Our 
definitions of g-analogs of weight multiplicities can be used for any choice of simple roots. Nevertheless, 
the results in this paper are based on the distinguished simple roots or on any set in their orbit under 
the action of the Weyl group of go. For other choices, minor modifications are required in the statement 
of some of our results. We will now specify the distinguished simple roots, cf. [8, List of Tables]. For 
gl(n, m) with n, m > 1, they are 

Sn~^ll^^ %^'5T' "^T^^l' ^1-^2, S„i-i-dm- 

For spo(2n, 1) with n > 1, the distinguished simple roots are 

~ '^TTT ' ■ ■ ■ ' % - <^T ' '^T • 
For spo(2n, 2to + 1) with n, m > 1, the distinguished simple roots are 

^ ^ ' • ■ ■ ' h~ h ^ Sj- Si , Si - $2 , . . . , (5„i_l - Sjn , Sjn- 

For spo(2n, 2) with n > 1, the distinguished simple roots are 

Si-Sn, Sn-S:f^, %-(5y, 2Si. 
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For spo(2n, 2m) with n > 1, m > 2, the distinguished simple roots are 

5n — ' • • • ' % ~ ^ > ^ — '^1 ) 5\ — 52 , . . . , 5m-l — 5m , <5m-l + 5m ■ 

We give below the sets of positive roots for Ql{n, m) and spo(2n, M) corresponding to the distinguished 
simple roots. 



g[(n, m) , n, m > 1 




6j — 6j\l<i<j<n 1 
5r — Ss\l<r<s<m J 






' . 1 1 < i < n ' 
''''1 1 < r < m J 


\ 


spo(2n, 2m + 1) , 
n > 1, m > 


Ao+ = < 


' 26j\l<i<n 
Sj±ST\l<i<j<n 
Sr \ 1 < r < m 
Sr±Ss\l<r<s<m 


> 






spo(2n,2) , n > 1 


A+ = < 


' 2S-\l<i<n 1 
6j±6j\l<i<j<n J 


At = {5i±Sj\l<i<n} 


spo(2n, 2m) , 
n > 1, m > 2 




' 2(5- 1 1 < i < n 

Sj ± Sj \ 1 < i < j < n 
dr±6s\l<r<s<m 


\ 




^5,±5r\ ^ 
1 <r <m 


\ 



Here, as well as throughout the paper, we adopt the convention that a set of elements which depend 
on some index in an empty range is empty. For instance, in the case of 5po(2n, 1) we have 

A+ = {25j, 6j±5j\l<i<j <n}, A+ = {5j \ 1 < i < n} . 

Denote by p+ and p_ the half sum of positive even and positive odd roots, respectively. Then set 
P ■■= P+-P- 

For any /? e [)* such that (3 = l3^%+--- + j3[°^5j+ 5i + --- + /J^^^ 6m , we will write /3 = 

where = . . . and /^^^^ = . . . For any k = (ki, . . . , Kp) e define |k| := 

Ki H h Kp- We denote by W the Weyl group of gO) and write I for the corresponding length function. 

For any w; e W, we set e{w) := {—1)^^^\ The Weyl group acts on I)*. As usual, for any w G W and 
/3 G [)*, we denote by w(/3) the action of the element w on /?. The dot action of W on 1^* is defined by 
w o (3 := w{P + p) — p. 

2.2. Classical root subsystems. The set of positive even roots Aq" is the set of positive roots for 
the Lie algebra Qq. Moreover, go can be identified with a direct sum 0„ ® Qm of two Lie algebras of 
classical type with ranks n and m. This splitting is shown in the table below for the superalgebras we 
are interested in; in all cases, we have n,m> 1. 






0l(n,m) 


spo(2n, 1) 


5po(2n, 2m + 1) 


5po(2n, 2m) 


00 = 0n ® 0m 


0t(n) ®0[(m) 


sp(2n) 


sp(2n) ®so(2m + 1) 


sp(2n) ® so(2m) 



Write A+ and A+ for the sets of positive roots of 0„ and Qm, respectively. Then Ag = A+ U A+. Let 
Wn and Wm be the Weyl groups associated to the root systems of 0„ and 0m- They contain subgroups Sn 
and 5m isomorphic to the symmetric groups of rank n and m. Write e*^") and e^^^ for the signatures defined 
on Wn and Wm- We have W = Wn x Wm, and e{w) = s{w^°^) e{w^^'>) for any w = (w^"^ ; w^^^) G W. 
The half sums of positive roots and p^^ of A+ and A+ verify p+ = (/9^^;p^^). We define the dot 
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action of Wn on the dual Cartan subalgebra of 0„ by u o t]^'^'' u {rj^'^^ + p^'^ — p^''. The dot action 
of Wm on the dual Cartan subalgebra of is defined similarly. 

Let Pn and Pm be the sets of integral weights of 0„ and Q„n respectively. Write P^ and P,^ for the 
subsets of dominant integral weights of P„ and P,„. Let P{n,m) C ()* be the set of integral weights of 
g, which can be regarded as the Cartesian product of P„ and Pm- Denote by P'^{n,m) C P{n,m) the 
subset of dominant integral weights of g, namely the set of weig hts A = (A(");A(i)) e P(n,m) such that 
A(°) e P+ and A(i' e P+ (see Remark [2X1]) . 

For the sake of completeness, we recall the explicit description of the weights A^^' = (^^r'' • • • ' ^j'') 
in P+ and A^^^ = • • ■ , Am^) in P+. In the case of gl{n,m), we have X^^ G Z for /c = 1, . . . ,n, 

and A-^^ > . . . > Xj ^ ■ The condition for A*^^^ is completely similar. In the case of spo(2rt, Af ), we have 
Ai°' G Z for A: = 1, . . . ,71, and Ai"^ > . . . > aI"^ > 0; thus, A(") is a partition with at most n parts. 
Similarly, A^^-* for fc = 1, . . . , m are either all in Z or all in ^ + Z; we also require A^^^ > . . . > A^^ > if 



S02m+i, and A^^^ > . . . > x';l^_^ > 



2 

> if Q,n = 502m- 



The dominant weight A G P^{n,m) is called typical if 

( A + p, a ) 7^ for any odd positive root a G , 

where :— Ai for g — g[(n, m), and A^'^ := {Sj ± | 1 < i < n, 1 < r < to} for g spo(2n, 2to + 1) 
and g = spo(2n, 2to). A dominant weight which is not typical is called atypical. 

Let VFstab be the largest subgroup in W which stabilizes A^. 

Remarks 2.2.1. (1) For g = g[(n, to) and spo(2n, 2), the set Aj'" is stable under the action of the Weyl 
group W. Therefore, we have Wstab — W. 

(2) For g ~ 5po(2n, 2to + 1) with to > and g = spo(2n, 2to) with to > 2, the set of odd positive 
roots Aj'" is not stable under the action of W. In this case, we have M^tab = Sn ^ Wm C W. 

2.3. Typical representations and character formula. To each A G P^{n,m) is associated an irre- 
ducible g-module of highest weight A that will be denoted by T^(A). In the sequel, we will only consider 
dominant weights for which V{X) is finite dimensional. We write Pj'{n,m) for the subset of P^{n,m) 
consisting of such weights. For g = Ql{n, to), we have P^{n, to) — P^{n, to). 

Remark 2.3.1. In [llj . the condition that ^(A) is finite dimensional is incorporated in the definition of 
a dominant weight. 

For g — spo(2n, M), the dominant weight A = (A*^"-*; A*-^-*) must verify an additional condition in order 
to guarantee finite dimensionality of V{X). In order to explain it, we use the explicit description of 
XiQ) ^ (^x!^)^ . . . , Ai°^) and A(i) = (A^^', . . . , A^^) in Section O Given this description, the irreducible 
spo(2n, M)-module V{X) is finite dimensional if and only if 

(1) aJ-^^ = for any j > Af \ 

In particular, llJ is verified when A^^^ > to. 

The module V{X) is called typical when A is typical, and atypical otherwise. For any integral weight 
p G P{n,m), write V^(A)^i for the weight subspace of weight p in V{X). Then K\ f^ — dimF(A)p is finite. 
The character of V{X) is defined by 



charF(A):= ^ K^.^e''. 
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Set 

Kac has proved that there exists an analog of the Weyl character formula for the typical finite dimensional 
simple modules V{X). 

Theorem 2.3.2. Consider X ^ (71,171) a typical dominant weight for q. Then 



char V{\) = V ^ 



Now define the partition function V based on the expansion 



(2) n ^ 

As a corollary of Theorem I2.3.2i we deduce that the multiplicities K\^^ for a typical weight A can be 
expressed in terms of the partition function just like in the case of the semisimple Lie algebras. 



Corollary 2.3.3. Consider A € P^{n,m) a typical weight and ji G P{n,m). Then 

Kx^f, = ^ e{w) V{w o A - ^). 



Remarks 2.3.4. (1) When A is atypical, the character formula of Theorem 12 . 3 . 21 does not hold in general 
(see [2]). The problem of determining the characters of the atypical irreducible g[(n, m)-modules has 
been intensively addressed in the literature. There exist in this case numerous character formulas of 
Weyl type appropriate to some particular atypical dominant weights [H |4l [6l [7l [TOl [121 HIl [H] • Unfor- 
tunately none of these formulas is known to hold in full generality. A general algorithm for computing 
the characters of the atypical irreducible g[(n, ?7i)-modules has been first given by Serganova [22l[23]. In 
[S], Brundan has also linked the computation of these characters to the determination of the canonical 
bases of certain [/q(g[(cxD))-modules. 

(2) For Q — g[(n, m), we have w{p-) — p- for any w S W. Hence, for any (3 G P(n,m), we have 
w o (3 = w{P + p+) — p+; in other words, p can be replaced by /?+ in the dot action of W . Similarly, 
when Q — 5po(2n, M), we have w o (3 = w{j3 + p-^) — for any [3 G P{n, m) and w e Wstab- Observe 
that the last statement does not hold in general for w £ \ VFstab- 

(3) By restricting from g to go, we can see that i^A.^ = Kx,w{fj.) for any w € W. 



Another important consequence of Theorem l2.3.2l is a branching rule for the restriction of the typical 
module ^(A) with A S P^{n,m) from g to go- For any 7 S P^[n,m), write ^^"(7) for the irreducible 
finite dimensional go-module of highest weight 7, and set 

mx^^ [V{\):V^-{^)]- 

namely, m\ ^ is the number of irreducible components isomorphic to V^°{'~^) in the restriction V{X) jgo- 
Set 

(3) Y{{l + en= 

Proposition 2.3.5. (c/. Proposition 2.11 in [11') Consider A 6 P^{n,m) a typical weight and 7 € 
P^{n,m). Then 

Jnx.-y — e{w) c{w o A — 7) . 
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2.4. Irreducible covariant tensor modules for m). In this paragraph, we consider g — gl{n, m). 
Then the set of positive odd roots can be written = {a^^^ \ < i < n,l < r < m}, where 
oti^r '■= — Sr- The superalgebra Ql{n, m) admits a natural irreducible module V with dimension m + n 
and highest weight 6n; this module may be atypical. For any positive integer k, the tensor power V'^'' is 
completely reducible |24| . Moreover the irreducible modules appearing in its decomposition have highest 
weights of the form A — {X'^^^; A^^^) such that 

• \X(0)\ + \xW\^k; 

is a partition with n parts (possibly equal to 0); 

• both A'-^-' and its conjugate /i := (A'-^-')' have at most m parts; 

• Ml ^ -^min' where ^ = (^i, ■ ■ • iMm); and a|^.j^ is the smallest nonzero part of A^"^ if A'°) ^ 0, 



whereas A 



(0) 



otherwise. 



These highest weights can be identified with (n,m)-hook Young diagrams by associating to the pair 
A = (A*^°^; A'-^-') a diagram Y{X) obtained by juxtaposing the Young diagrams of A^"-* and /i = (A*^^^)' as 
illustrated in the example below. The combinatorics of hook Young diagrams was first studied in [2] . 

Example 2.4.1. The hook Young diagram Y{\) for A^''^ = (9, 7, 5) and A^i) = (4, 3, 3, 2) (thus (n, m) = 
(3, 4) and /i = (4, 4, 3, 1)) is indicated below. 

m I 



m 



Observe that there is no box at the intersection of the [n + l)-th row and the {m + l)-th column. By 
the convention stated earlier, we can write A — + 7% + hSj + 4(5i + 3(^2 + 3(53 + 2(52- 

The tensor power contains typical and atypical irreducible components. Define 
Y^{n, m) := |a = (A^"\ A'^^) | V{X) appears as an irreducible component of a tensor power V^®*^! . 

Set Y+( 

Consider A G Y^{n,m). Set 

Vo:=e-^+ n (l_e-")-i, 

and write A^j^ for the subset of A^ containing the roots ai^r such that Y{\) has a box at the intersection 
of its i-th row and its r-th column. There exists a character formula of Weyl type for V{X) due to Berele- 
Regev |5] and Sergeev 

Theorem 2.4.2. The character of the covariant tensor module V{\) is given by 

( 



char V^(A) Vq ^ e(w) - 



v 



+ 
l,A 
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Set 

(4) n + E 

aGA+^ KGP(n,m) 

We deduce from the previous proposition the foUowing expression for the multiphcities mx^j. 

Corollary 2.4.3. For any X G y+(n,m) and any 7 G P^{n,m), the multiplicity m\^j ofV^"{'y) in 
V{X) is given by 

"T-A,7 = E ^^'^^ ^'^^"^ - w o 7) . 

Remarks 2.4.4. (1) When A^*";^ ~ A^, the partition functions c and ca coincide. Moreover, we have 
c{w{k)) = c{k) for any k e P{n, m) and any w ^W. Thus the expression of the multiplicity m^.^ in the 
Corollarv 12. 4. 31 coincides with that in Proposition 12.3.51 

(2) We cannot derive from the character formula in Theorem 12.4.21 a simple expression for the multi- 
plicity K\,^ similar to the one for A typical in Corollarv l2.3.3l 



3. LUSZTIG g-ANALOGS AND g-PARTITION FUNCTIONS 

3.1. Lusztig q-analogs for the Lie algebras 0„ and g^. Denote by 'Pn,q and Vm.q the g-partition 
functions defined by 

llaGA+U 96 ) ^(o)g^„ llaeA+U 96 ) ^^^^^^^^ 

Consider two weights 7 = (7*-°^; 7^"'^^) and ji — (/i^"-* ; /i^"'^') in P{n,m). Set 
These polynomials are Lusztig q- analogs for the Lie algebras g„ and g„i. 

Theorem 3.1.1. (Lusztig |20j ) Assume that j,fi ^ P^{n,m). Then the polynomials K^"^^ f^mil) '^'"^^ 
K^O-) fi(i)il) have nonnegative integer coefficients. 

We will also need the stabilized form of the Lusztig g-analog corresponding to 0„. We define 
(7) ^Xto)(9) E e^"\u)VnAnoj^'^-t.^'^). 

Observe that the sum runs over the symmetric group on n letters. For g = gl("i '^): '^6 have g„ ~ oK"-) 
and thus -?^^"of '^(o) (9) — ^^"o) f^mil)- For g — 5po{2n, M), we have g„ ~ sp2„- Write for the n-th 
fundamental weight of g„. We can identify ui with the weight (1, . . . . 1; 0, . . . , 0) e P^{n,m). The 
following lemma has been proved in [18j . 

Lemma 3.1.2. For any integer k > ^ LJ have ^^ro)+fc^,p(o)+fe^('7) = ^^m^^) (?)• pariic- 

uZar, X^^roOo, (9) e Z>o [g] when 7(0) , e P+ . 

3.2. The Lusztig g-analog for go- We define the g-partition function Tq similarly to ([5]), by 
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Definition 3.2.1. For any 7,// G P{n,m), the polynomial K^°^{q) is given by 

^i^M - J2 ^^(^('^ + p+) - M - p+) . 

The stabilized polynomial i^T^"^"**^^ (q) is given by 

Since w{p-) = p- for any w 6 W^stab, one can also replace p+ by p in the latter formula. 

For a dominant weight 7, the polynomial KS^^^j^iq) is a g-analog for the dimension K^°^ of the weight 
space p in the go-module of highest weight 7. From the considerations in Section [221 Theorem 13. 1.11 
and Lemma I3.1.2[ we deduce easily the following proposition. 

Proposition 3.2.2. For 7 — (7^°-'; 7^"'^-') and p — {p^^'' ; p'^^^) in P{n,m), we have, with the above 
notation, the factorizations 

In particular, = K^»if^''^{q). Furthermore, K^o^{q) and KS^"if^'''°{q) belong to 'L>o[q] when 

7 and p are dominant. 

Consider ^ e P{n, m). There exists a straightening procedure for the polynomials K^°^(q). 

Lemma 3.2.3. Consider p G P^{n,m) and ^ G P{n,m). Then 

K^" (q) = I ^^''^ ^I'Al) = + P+) - P+ reW and-f^ P+{n, m) 
1 otherwise. 

In particular the coefficients of K^°^(q) are integers with the same sign. 

Proof. The proof follows easily from the equality -^t(7+p+)-p+ ~ ^(''') -^^"nil) ^'^^ ^ '-' 

Remark 3.2.4. Set 

For g = gK*^! ™) and 5po(2n, 2), stabihzes the set of positive odd roots A^. Thus we have A?^°^{q) = 
K^°^{q). This equality does not hold for the other superalgebras spo{2n, M). This will cause some 
complications in Section [5l 

3.3. A q-partition function associated to g. We define the following natural g-analog, denoted by 
Vq, of the partition function P in ([2]). This g-partition function will be used below to define our g-analogs 
of weight multiplicities for the Lie superalgebras g[(ri, m) and spo(2ri, M). 

Definition 3.3.1. The q-partition function Vq is given by 

Let us justify why Vq is a natural g-analog, in the spirit of the g-partition functions in due to 
Lusztig. As Kac pointed out in [TJ, V{l3) counts the number of ways to express /? as J2aGA+ ''^^ct with Ua 
in {0, 1} for a in A^, and in Z>o for a in A^j". Thus, the sequences n — {na)ai£A+ are generalizations 
of the Kostant partitions relevant to We can define the length of the Kostant partition tt, as usual, 
by £(7r) := X]aeA+ Then Pq{P) = J2w1^^'^^^ precisely as the g-partition functions in (O can be 
expressed. 



(j-ANALOGS OF WEIGHT MULTIPLICITIES FOR LIE SUPERALGEBRAS 



9 



We define the q-partition function Cq similarly to ([3]), by 

(9) [](l + qe")= 

aeA+ K6P(n,m) 

Then, for any /3 £ P{n,m), we have 

(10) Vg{P)= Y c,(«)^,(^). 

4. Some g- analogs of weight multiplicities for Ql{n,Tn) 

In this section we consider g = gK^-i '^)- For any dominant weight A which is either typical or belongs 
to Y~^{n,m), and any weight fi e P{n,m), we introduce a g-analog Kx^fj,(q) for the weight multiplicity 
Kx,fj,. We prove that this q-analog has nonnegative integer coefficients when fj, G P'^ (71,111,). 

4.1. Typical modules. We start by defining K\_^{q) for typical modules similarly to 

Definition 4.1.1. Consider a typical dominant weight A and /i G P{n,m). The polynomial K\^^{q) is 
given by 

Kx,t,iq) := X! '^1^'^ ° A - Ai) , 

where is the g-partition function in Definition 13.3.11 

According to Corollary 12.3.31 we have Kx.fj.{l) = K\^^, so K\^^{q) is a g-analog for the multiphcity 
of n in V{X). We clearly have K\^^{q) G Z[g]. 

Now observe that for any a — {a'^^^ ; a'^^'^) G Aj^, we have |a'-*'-'| = 1. This implies that if k = 
(k'^*'^; k*^^^) G Pin, m) is expressed as a sum of distinct positive odd roots, then the number of summands 
is equal to or . Thus, with the notation in ([3]) and we obtain Cq{K) = gl''' ''Ic(k). 

By the definition ([5]) of the partition function Tq, we can have J-q{r]) ^ for some rj = (77^°); ry*^^^) G 
Pn,m only if 1 77^°) I = [jy'-^-'l = 0. Based on the previous observations, (fTO]) can be rewritten 

(11) Pg(/3) =gl'3""l ^ c{K)TqiP~K) forany/3= (/3W;/3(i)) GP(n,m). 

KGP(n,m) 

The following theorem can be regarded as an analog of Theorem 13.1.11 for the Lie superalgebra gl{n, m). 
Theorem 4.1.2. Consider a typical dominant weight A and fj, in P{n,m). Then 

(12) i^A,p(g)-gl^""l"l^""l Y ™A,7^^7(o),;.(o)(g)i^^(i),^(i)(g). 

(n.m) 

In particular, K\^^{q) belongs to Z>o[g] when fi is dominant. 
Proof. We derive from (fTTj) the equality 

^A,^(g) = ^ £(u') Y ^('*) 9'''""' ^gCw'lA + p) - (Ai + K + p)) , 

where /3 = u>(A + p) — (/i + p) in the second sum. This notably implies that = |'^'-''-'| ~ since 

W = Sn Srn- So wc obtain 

i^A,p(g) = gl^'"'|-|^'"'l ^ e(z«) 5] c(^^).F,(^A + p-^«-l(^^))-(A^ + p)). 
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For any w ^ W and any k G P{n,, m), we have c{k) — c{w{k)) because is stable under the action of 
the Weyl group W. Thus, by setting ^ :— in previous expression, we can write 

weW CeP(n,m) 
{eP(n,m) 

where the polynomials Kx'L^ ^(g) aic those in Definition l3.2.1l Now by Lemma [3.2.3[ we have K^''_^ ^{q) — 
0, or there exists w € W and 7 G P^{n,m) such that 7 = w^^ o (A — ^). In the latter case, we have 
= X + p — w{'^ + p) , and it follows that 

lueW 7GP+(n,m) 

Since c(^) = c(u;^^(f)) for any w ^ W and ^ G ^'(n-, rri), we obtain 

76P+(n,m) ujSW 

Now by Proposition 12.3.51 we derive 

7e^P+ (n,m) 

Finally, the desired equality and the positivity for p dominant immediately follow from Proposition 

□ 

Remark 4.1.3. Recalling the q-partition functions Cg and J-q defined in ([9]) and ((8]), we note that, in 
practice, it is useful to calculate K\^i^i{q) by combining Definition 14.1.11 with PH)) . as follows: 

(13) Kx.j^iq) ^ ^ £{w) Cq(K) Tq{w O \ - p - k) . 

luGW K£P{n,rn) 

Recalling the notation in ([5]), also observe that 

We implemented a Maple procedure based on this approach, which is part of a package made available at 
math.albany.edu/math/pers/lenart/ and lmpa.univ-littoral.fr/~lecouvey/. The package also 
contains procedures which compute the g-analogs for the orthosymplectic superalgebras defined in Sec- 
tion [5l The partition functions Pn.q and Vm.q are computed by using Gelfand-Tsetlin patterns of type 
A — D ^ to represent Kostant partitions, and by generating these patterns recursively. An efficient 
implementation is used in order to reduce the computational complexity. 

Example 4.1.4. Using our Maple procedure, we computed if(3.i,_2;4,2,-8),(o,o,0;0,o,o)(9) for sK^jS): 
2q22 + 8q2i _^ 22g20 + 40q^^ + 57q^^ + eiq^^ + 52q^^ + 33q^^ + IGq" + 5q^^ + q^^ . 

Recall that a polynomial is called unimodal if the sequence of its coefficients has the corresponding 
property. Based on our experiments, we make the following conjecture. 

Conjecture 4.1.5. For any typical dominant weight A and p G P^{n,m), the polynomial K\^{q) is 
unimodal. 

According to Thcorcm l2.3.2( we can also define the graded character of the typical module V{\) by 

(14) char, V{\) V(g) ^ e(w) e^^^+P^ , 
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where 

n«A;(i-««-°)' 

The coefBcients of the expansion of char^ V{X) in the basis of formal exponentials also yield g-analogs 
of weight multiplicities. It is easy to verify that these g-analogs coincide with the polynomials Kx^^ici), 
namely we have 

char,F(A)= ^ K^^.M^'^- 

By Theorem 14.1 .21 we then obtam the fohowing expression for char^ V^(A): 
(15) char,F(A)= ^ hk'"' I ^ m,,^ 

4.2. Irreducible covariant tensor modules. For v a typical dominant weight, it is possible to define 
the q-analogs Ki,_^{q) directly from CoroUarv 12.3.31 or by introducing the graded character char, V{i^). 
Now assume A G F+(n,m). In this case, we have seen (cf. Remark 12.4.41 (2)) that there is no analog 
of CoroUarv 12.3.31 for the multiplicities Kx^f^. Therefore, we define a graded version of the character in 
Theorem [5X1 

Definition 4.2.1. Let 

/ \ 

char, V{\) := 'Vo{q) ^ e{w)w 



where 



aeA + 

Based on the above graded character, we define the g-analog in the obvious way. 
Definition 4.2.2. The polynomials K\,^{q) are given by 

char,F(A)= KxM)^""- 

Clearly, Kx,^{l) is equal to Kx,fj,. 
Theorem 4.2.3. We have 

In particular, the polynomials K\^^{ci) also verify p^ . and thus have nonnegative integer coefficients 
when fi G P^{n,m). 

Proof. The arguments of the proof are close to those used in the proof of Theorem l4.1.2l By we can 
write 

By the definition ([5]) of the g-partition function Tq, we then derive 

char,F(A)- Y E E =H 'cAM.^,(Oe'^(^+''+-^)-«-''+. 
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By setting /i w{X + p+ — k) — ^ — p+, this yields 



char,F(A)= ^ q\''^°^\ cxin) ^ ^ e(u)) jf<,(u;(A + /?+ - k) - /i - p+) e''. 
Thus, we derive 

char,F(A)= E q^'''°'^cx{^)Kr-.Jq)e^- 

By Lemma r3.2.3l we have if^l^ ^{q) — 0, or there exists w G W and 7 G P^{n,m) such that 7 = 
w~"'^(A + p+ — k) — Then we have k = A + p+ — 1/7(7 + particular, [k^"^ | = |A*-°'' | — j/i*-"'' | and 

it follows that 

char,y(A)= E E ca(A + p+ - u;(7 + P+)) 

/JG-P(n,m) 7S-P+ (n.m) u^giy 

Since A + p+ — w(7 + p+) = A — w o 7, this yields the theorem by using Corollarv l2.4.3l □ 



Remark 4.2.4. The methods used in this paragraph to define g-analogs of weight multiplicities corre- 
sponding to the irreducible covariant tensor modules can be extended to any highest weight g[(?i,TO)- 
module ^(A) whose character is given by a Weyl-type character formula of the form 



char y(A) Vo ^ w I 6^+"+ + 



where S^^ is a subset of depending on A (cf. [21]). The polynomials K\^^[q) are then defined as the 
coefficients of the expansion of the corresponding graded character in the basis of formal exponentials. 
They are also expressed as in p2)) . and therefore have nonnegative coefficients when /i S P~^(n, m). Such 
a situation also occurs, for instance, when ^(A) is a singly atypical module, that is, when there exists a 
unique root a\ in Aj*" such that { X + p, ax) — 0. In this case, we have S^^ — A'^ \ {a\} (see [6l [7]). 



4.3. Charge statistic on semistandard hook-tableaux. We now briefly recall some background 
on the quantum superalgebra Uq{Ql{n,m)) and the notion of a crystal basis (cf. •13]) introduced by 
Benkart, Kang and Kashiwara [1]. The reader is referred to the latter paper for a complete exposition. 
One associates to the Lie superalgebra gl{n, m) its quantized enveloping algebra J7g(g[(n, m)), which is a 
Hopf algebra. This algebra possesses an irreducible module Vq of dimension m+n. The tensor powers Vj®*^ 
for k S Z>o are completely reducible. The irreducible modules appearing in their decompositions into 
irreducible components are the highest weight J7g(£|[(n, m))-modules V;j(A) with A E Y^{n,m). To each 
A e F+(n, m) is associated a t/g(0[(n, TO))-crystal B{X). This is an oriented graph with arrows colored 
with the set of colors {n — 1, . . . , 1, 0, 1, . . . , m — 1}. The vertices of B{X) are labelled by semistandard 
hook-tableaux of shape A on the totally ordered alphabet An,m {n < ■ ■ ■ < 1 < 1 < ■ ■ ■ < to}. Here, 
by a semistandard hook-tableau of shape A, we mean a filling of the Young diagram Y{X) (see Example 
12.4. ip with letters of An,m subject to the following conditions. 

(1) The letters in each row are increasing from left to right. Repetition of barred letters is permitted, 
but repetition of unbarred letters is not. 

(2) The letters in each column are increasing from top to bottom. Repetition of unbarred letters is 
permitted, but repetition of barred letters is not. 
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Example 4.3.1. The following tableau is a semistandard hook-tableau of shape A — (A'^"^ A'^^-') for 
(n,TO) = (3,4), A(") = (7,6,4), and A^^) = (4,3,3,2). 



3 


3 


2 


1 


1 


2 


3 


2 


2 


1 


2 


3 


4 




1 


1 


1 


2 






1 


2 


3 


4 






1 


2 


3 


4 






2 


3 


4 








2 













The crystal i?(A) has also the structure of a C/q(go)-crystal obtained by deleting the arrows colored 
0. Write B°(X) for the crystal obtained in this way. Then, for any 7 G P'^{n,m), the multiplicity toa,7 
is equal to the number of connected components in B°{X) of highest weight 7. 

Write B^"{-f) for the abstract C/g(0o)-crystal of highest weight 7. Since go = sK") © flK™)? have 
BS''(7) = Bs'"(7('')) X B8'™(7(i)), i.e., we obtain the direct product of the C/q(0Z„)-crystal B^''^ {-f^°'>) 
and the ?7q(gZm)-crystal _Bb''"(7'^^)). In particular, the corresponding vertices are labelled by the pairs 
of tableaux (r(°),T(i)) such that T(°) (resp. T^^)) is semistandard on {n, . . . ,T} (resp. on {1, . . . ,to}) 
of shape 7*^°-' (resp. 7'-^''). In [TH], Lascoux and Schiitzenberger proved that the Lusztig g-analog corre- 
sponding to the general linear group can be expressed as the generating series for a special statistic ch 
on semistandard tableaux called charge (see [19^ for a complete exposition) . By the previous arguments 
and Proposition 13.2.^ this implies the following proposition. 

Proposition 4.3.2. Consider fi <E P^{n,m). Then 

(T(0),T(i))eBB0(7)^ 

where i?Bo(7)^ is the set of vertices in B^"{'-/) with weight fi. 

Now consider G B°{X) a highest weight vertex of weight 7. Then, the connected component B{T^) 
of B°{X) containing is isomorphic to B^°{'y). Let 6 be the corresponding isomorphism. Consider 
T G B{T^) and set 0{T) = {T'^°\t'-^^). It is easy to check that T^") is the tableau obtained by deleting 
the unbarred letters in T. To obtain T^^\ we first delete the barred letters in T. This gives a skew tableau 
that we can conjugate (i.e., reflect in a diagonal), in order to obtain a skew semistandard tableau T'. 
Then T*^^^ is obtained by rectifying T' via Schiitzenberger 's jeu de taquin (e.g., see [9l I15j). 

It is then natural to define the charge of the semistandard hook-tableau T by ch(r) :— ch(T(°)) -f- 
ch(T(i)), where T'"' and T^^) are obtained by the previous procedure. Since, the polynomials K\^fj_{q) 
with A G y+(n, m) verify (|12p . we obtain the following theorem. 

Theorem 4.3.3. Let A G Y^{n,m) and fi G P^{n,m) he two dominant weights. Then 

TeSSHT(A)^ 

where SSIIT(A)p is the set of semistandard hook-tableaux of shape A and weight /x. 

5. Some ^-analogs of weight multiplicities for spo(2r7,, M) 

In this section, we assume that g = spo(2n, 2to + 1) or g = spo(2ri, 2m). We introduce g-analogs 
K\,fj.{q) for the multiphcities K\^^ corresponding to a typical weight A G Pf'{n, m). Although the family 
of g-analogs K\^^{q) for such A and /i G P^{n,m) contains polynomials with negative coefficients, this 
family possesses a natural subfamily (the stabilized K\^fj,{q)) for which the positivity property holds. 
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5.1. The polynomials Kx ^^{q). We define the g-analog Kx ^^{q) similarly to Defimtion l4.1.1l 
Definition 5.1.1. For any weight A G P^{n, m) and any /i G P{n, m), the polynomial Kx ^{q) is given 

by 

Kx,f,iq) ■■= e{w) Vq{w o \ ~ ^) , 

where Vq is the g-partition function in Definition 13.3.11 

When A G P^{n,m) is typical, the polynomials Kx^^{q) coincide with the coefficients appearing in 
the expansion of the graded character 

char,, V{\) = V(q) e{w)e'^^^+p\ 

We have then Kx,tL{^) = K\^, and therefore K\^{q) is a q-analog for the dimension of the weight space 
/i in the finite dimensional module V{X). Observe that the hypothesis /i G P^{n,m) does not suffice to 
guarantee that Kx,^{q) belongs to Z>o[(i'], as illustrated by the following example. 

Example 5.1.2. Consider spo(2n, 2m + 1) for n = 1 and m = 2. We have p = (—2; 1,0) + h, where 
h = (i; i, i). We verify that A = (2; 1, 1) is in Pf{n, m), and let ^ = (0; 2, 1) in P'^{n, m). We calculate 
Kx,fj,iq) based on a formula identical to (|13p . as shown in the table below. The only weights k for which we 
have nonzero terms in the right-hand side of (fT3|) are (1; —1, 0), (2; —1, —1), and (2; —1, 0). The columns 
labeled u and v correspond to those w = (w; v) in W = Wn x Wm for which J-q{w o A — /i — k) / 0. In 
accordance with standard notation, sj is the generator of Wn- Also note that, if we view w as a signed 
permutation of 1, . . . , n + m, then h^, := w{h) — h is a sequence of — I's and O's, where we have a —1 in 
position i precisely when w contains i. 



K 


c,(k) 


ri:=woX — fi — K 


u 




V 




(i;-i,o) 


q 


ii;(0;2,l) + (l;-2,-l) + /i„ 


sj 


-1 


Id 


1 


(2; -1,-1) 


q' 


w(0;2,l) + (0;-2,0) + /z^ 


Id 


1 


Id 


q 


(2; -1,0) 




w(0;2,l) + (0;-2,-l) + /i,„ 


Id 


1 


Id 


1 



Hence, we have i^(2:i,i),(0:2,i)('?) = q^ + q^ - q. Another example is if (5,4,4;3,2,o),(3,2,i;i,i,o)('?) for 
spo(2n, 2m + 1) with n = m = 3, which was computed with our Maple procedure: 

iq^^ + 1^0 + 52^29 _^ -^4g^28 ^ 373^27 ^ 3^7^26 _^ i640(?25 + ZQQQq^^ + 5132q23 + 8174q22 

+ 12283(?2i + 17338g20 + 23138gi^ + 28977gi* + 34022gi^ + ZQmZq^^ + 36953(7^^ + 332599^'' 
+ 26478q" + 18045g^2 + 10121g" + 4332gi" + 1211q^ + 97g* - 65g^ - 17q® + q^ . 

5.2. Stabilized q-analogs. We now define the subfamily of the polynomials Kx,ii{q) for which the 
positivity property holds. 

Definition 5.2.1. For A G P^{n, m) and /i G P{n^ m), the polynomial Kfi'^^{q) is given by 

Ktf{q)--= E e{w)Vq{wo\-^,). 

Note that the sum in the definition of Kf''J^{q) runs over Wstab, and A is not assumed to be either 
typical or in P^{n, m). Moreover, we have Kf!^ ^+i^{q) = ^f^^iq)^ where lu = (1, . . . , 1; 0, . . . , 0) G 
P^{n,m). This justifies our terminology. 

Lemma 5.2.2. Given A G P^{n,'m) and fi G P{n,ni), there exists fco G Z>o such that, for any 
nonnegative integer k >ko, the following conditions are verified: 

(1) A + few G PUn,m); 
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(2) A + kuj is typical; 

(3) K^+k.,^+Uq) ^ Kf fi'i)- 

Proof. We will assume that g = 5po(2n, 2m + 1). The case g = spo(2n, 2m) is similar. 

(1) Write X + kuj^iy^ iy^^)). Then 4"' = a["^ + k. Thus, by we have e m) for any 
k such that A^^^ + k >m. 

(2) We have p = (n — m — i, . . . , i — m; m — i, . . . , i) and Aj^ = ± Sr \ 1 < i < n,l < r < m} . 
Thus, for any i G {1, . . . , n} and r £ {1, . . . , m}, we have 

( + p, 5,- - (5r ) = Af ^ + A[^' + n + fc-?;-r+l and 

{i^ + p, 6j+Sr) = Af^ -Afi) +n-2m + fc-i + r. 

It is then possible to choose k sufficiently large so that {v + p, 6- — Sr) > and {u + p, d-+ S,.) > for 
all positive odd roots Sj — Sr and Sj + 6,- . 

(3) By the definition of the g-partition function Tg, we have J^g{(3) = when |/3(°)| < 0. Consider 



w 



— (u''''^ w'^^) e W such that w ^ Wgtab- Since w'"^ ^ ^n, the signed permutation w^^^ changes the 



sign of at least one coordinate in v'^^^ + p'^^\ Thus we have 



< 



A(" 



2A;. 



Then for any integer k > LJ -^re have J-q{w o v — p) = 0, and therefore K\j^k^,f^+ku){q) = 

Kff[q). □ 

Remarks 5.2.3. (1) Penkov and Serganova [21] gave a character formula for a simple Lie superalgebra 
and a corresponding generic weight A. Their definition of a generic weight is analogous to the conditions 
(l)-(3) in Lemma [OH 

(2) By Lemma 15.2.21 for any A G P+(n,m) and any p G P{n,m), the polynomial K^^{q) can 
be regarded as a q-analog for the multiplicity of the weight p + kauj in the finite dimensional module 
V{X + kouj). 

(3) Suppose that A G P^{n,m) and p G P{n,m) are such that i^A+w,/i+w('z) = K\^^{q). Then for any 
nonnegative integer k, we have Kx+kuj,f,+kij{q) = K\Al)- Thus Kx,f,{q) = Kf '^J^iq). 

Theorem 5.2.4. Consider A, G P^{n,rn). Then Kf f{q) belongs to Z>o[g]. 

The proof uses the following easy lemma. 
Lemma 5.2.5. Consider A,7 G P^{n,m) and let 

^fn- eH c{w{\ + p+) - - p+) . 

Then mf^^ G Z>q. 

Proof of Lemma \5 .2 .5[ We observe that c(k) = for any k G P{n,m) such that < 0. By using 

arguments similar to those in the proof of Lemma 15.2.21 (3), there exists a nonnegative integer k such 
that mf'^^ = mx+kLo,-i+kuj. □ 

Proof of Theorem \5. 2. 4[ From the description of Aj^ and ([TUj), we obtain 

W)^ <l^"''"'^c{K)Tg{P-K) for any /3 = (/3(°) GF(n,m). 

/tGP(n,m) 
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Recall also that w o (3 = w{P + p^) ~ p-^- for any w e W^stab- By Definition 1 5. 2. 11 this implies that 
Ktfi^)- E e{w)T,{w{X + p+)-{p + K + p+)). 

Hence 

KT^iq)^ E E q\''""\c{K)T,{w{x + p+-w-'{K))-{p + p+)). 

■MGWatab KGP(T!,m) 

Set ^ = w~^{k) in the previous sum. Since w G Wstab, we have c(k) — c(^) and = \^^^^\: because 

is stable under the action of H^stab- Thus we can write 

Kfil)- E =H E 9l^'"lc(0^,MA + p+-0-(/i + P+)) 

where K^"^^^'^ (q) are the polynomials in Definition l3.2.1l Now, by Lemma [3.2.31 we have K^^S^^^^ (q) — 0, 
or there exists w G Wstab and 7 G P^{n,m) such that 7 = ti;^^(A — ^ + p+) — p+. Hence, we have 
C = A + p+ - u;(7 + and | = | A(°) | - 17(0) | . Thus 

Ktt{q)= E E '?l''"|-|"'"'lc(A + p+-^7 + P+))^rM^*"'('?). 

Wstab 7GP+(n,m) 

Since c(^) = c{'w~^{^)) for any ?i; G Wstab and ^ G P{n, m), we obtain 

Kt^iq)= E gl''"''"'^""' E £HcMA + p+)-7-p+)if«-^*^^g). 

7eP+(n,m) lueWatab 

Finally, one derives the expression 

7GP+ (n.m) 

By Proposition 13.2.21 and Lemma [5. 2. 5[ this implies Kf-^{q) G Z>o['z] when /i G P^{n,m). □ 

Corollary 5.2.6. Consider A, /i G P+(n, m). There exists an integer kg G Z>o smc/i that, for any integer 
k > kg, the following assertions hold: 

(1) A + fccj G P/( n, m); 

(2) A + kuj is typical; 

(3) Kx^ku,p,+kLj{q) has nonnegative integer coefficients. 

Remark 5.2.7. We have already observed that for g — spo(2r7,, 2) we have W — Wstab- This implies that, 
for any A G Pj'{n, m) and any p G P^{n, to), the polynomial Kx,i_i{q) = K^'^{q) has nonnegative integer 
coefficients. 

Example 5.2.8. This is a continuation of Example 15.1.21 We first considered A = (2; 1,1) and p = 
(0; 2, 1) for spo(2n, 2m + 1) with n — 1 and m = 2. The smallest fc > for which A + fccj is typical is 3. 
So let us consider X' := X + 3u! = (5; 1, 1) and p' := p + 3uj — (3; 2, 1). It turns out that 

'^(2rM),(0;2.1)('Z) = ^(5;14),(3;24)(9) + 

Indeed, the case corresponding to k = (1;— 1,0), which is illustrated on the first row of the table in 
Example 15.1.21 and which produced the negative term —q, does not appear anymore. To be more 
precise, in this case we have w o X' — p' ~ k — w{3; 2, 1) + (—2; —2, —1) + h^; but for w = (s-; Id) this is 
(-6; 0,0), and clearly -6; 0, 0) = 0. Next, we considered A = (5, 4, 4; 3, 2, 0) and p = (3, 2, 1; 1, 1, 0) 
for spo(2ri, 2m + 1) with n = m — 3. In this case, the smallest fc > for which A + fco; is typical is 5. 
Based on the computations with our package, we have 

^{5^i,4:;3,2,0),{3,2,l;l,l,0)iq) = ^(10,9,9;3,2,0),(8,7,6;1,1,0) (?) : 
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and this polynomial is 

3q^^ + Uq^° + 52g29 + U8q'^» -|- 373^27 _,_ 820^26 + 1655?^^ + 3052q^'^ + 5266?^^ + 8475g^^ 
+ 1287V^ + 18421g^° + 24941g^^ + 31772g^* + 38048?^^ + 42412g^® + 43722q^^ + 41083(7 
+ 34742gi3 _^ 25932q^^ + 16776g" + 9175q^° + 4129g^ + U76q^ + 395q^ + 70q^ + 6q^ . 

As in the case of m), based on our experiments, we make the following conjecture. 
Conjecture 5.2.9. Given A,/i e P^{n,m), the polynomial Kf^{q) is unimodal. 
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